Active Rearranged Capturing of Image-Based Rendering Scenes-Theory and Practice

In this paper, we propose to capture image-based rendering scenes using a novel approach called active rearranged capturing (ARC). Given the total number of available cameras, ARC moves them strategically on the camera plane in order to minimize the sum of squared rendering errors for a given set of light rays to be rendered. Assuming the scene changes slowly so that the optimized camera locations are valid in the next time instance, we formulate the problem as a recursive weighted vector quantization problem, which can be solved efficiently. The ARC approach is verified on both synthetic and real-world scenes. In particular, a large self-reconfigurable camera array is built to demonstrate ARC’s performance on real-world scenes. The system renders virtual views at 5-10 frames per second depending on the scene complexity on a moderately equipped computer. Given the virtual view point, the cameras move on a set of rails to perform ARC and improve the rendering quality on the fly. Index Terms Active sampling, active rearranged capturing, image-based rendering, camera array, self-reconfigurable. Manuscript received xxx xx, 2005; revised xxx xx, 200x. Work performed at CMU and supported by NSF Career Award 9984858. C. Zhang is with Microsoft Research, Redmond; T. Chen is with Department of Electrical and Computer Engineering, Carnegie Mellon University. September 7, 2006 DRAFT IEEE TRANS. ON MULTIMEDIA, VOL. X, NO. X, XXX 200X 1 Active Rearranged Capturing of Image-Based Rendering Scenes — Theory and Practice